1. Field of the Invention
The present invention relates to a recognition unit and a recognition apparatus having a plurality of recognition units organized in a multilayered hierarchical structure. The recognition apparatus is preferably utilized as a learning apparatus capable of recognizing an object according to various characteristic data thereof through learning.
2. Description of the Prior Art
A conventional learning apparatus is disclosed, for example, in "Learning Representations by Back-Propagating Errors" written by D. E. Rumelhart, G. E. Hinton, and R. J. Williams and published in "Nature, vol.323, pp.533-536, Oct. 9, 1986". This learning apparatus is schematically shown in FIG. 1.
As shown in FIG. 1, the learning apparatus comprises an output signal calculating section 10 and a weight renewing section 20. The weight renewing section 20 renews the value of weights of the output signal calculating section 10 based on an output signal from the output signal calculating section 10.
The output signal calculating section 10 is organized in a hierarchical structure as shown in FIG. 2. As clearly shown in FIG. 2, the output signal calculating section 10 comprises a plurality of input sections 40 and a plurality of signal processing sections 30. Each of the signal processing sections 30 derives at least one output from a plurality of inputs.
As shown in FIG. 3, the signal processing section 30 comprises a plurality of input sections 50, a memory 60 in which are stored a plurality of weights for weighting inputs from respective input sections 50, a plurality of multipliers 70 each for multiplying an input from each input section 50 by a weight stored in the memory 60, an adder 80 for adding outputs from the multipliers 70, and a threshold processor 90 for limiting an output from the adder 80 to a value falling within a given range.
FIG. 4 is a graph indicating an input/output characteristic function of the threshold processor 90, which is given, for example, by: ##EQU1## where I is an input to the threshold processor 90. According to this equation, the output from the threshold processor is limited to a value falling within the range of (0, 1). The input/output characteristic function shown above may be replaced by any other suitable threshold function.
Referring back to FIG. 1, the weight renewing section 20 comprises a teacher signal generating section 100, an error signal calculating section 110, and a weight alteration amount calculating section 120.
The learning apparatus having the above-described construction operates as follows.
When the input sections 40 of the output signal calculating section 10 receive respective input signals, the multipliers 70 of each signal processing section 30 multiply outputs from the signal processing sections 30 connected therewith and located at a lower layer than the layer thereof by respective weights stored in the memory 60. The weights or loads are indicative of the strength in the connection between two signal processing sections 30. The sum of outputs from the multipliers 70 is calculated by the adder 80 and is converted by the threshold processor 90. Thereafter, the resultant value is outputted to one or more signal processing sections 30 located at the next upper layer.
More specifically, each of the signal processing sections 30 performs an operation given by: EQU o.sub.i =f(.SIGMA..sub.j w.sub.ij o.sub.j)
where o.sub.j is a value inputted to the input section 50 (an output from a j.sup.th signal processing section 30 at the lower layer), w.sub.ij is a weight stored in the memory 60 (a weight in the connection between an i.sup.th signal processing section 30 and the j.sup.th signal processing section 30 at the lower layer), and .SIGMA. is the sum of all the weights connected with the i.sup.th signal processing section 30.
According to signals inputted to the output signal calculating section 10 via the input sections 40 thereof, the teacher signal generating section 100 generates, as a teacher signal t.sub.i ("0" or "1"), an appropriate output signal. Thereafter, the error signal calculating section 110 calculates a difference (t.sub.i -o.sub.i) between the teacher signal and the signal o.sub.i actually outputted from the output signal calculating section 10. This difference is outputted to the weight alteration amount calculating section 120, which calculates a square error in the signal processing sections 30 at the uppermost layer from the difference (t.sub.i -o.sub.i). The square error is given by: EQU E=0.5.SIGMA..sub.i (t.sub.i -o.sub.i).sup.2
Based on the square error E, the weight alteration amount calculating section 120 calculates the amount of alteration of the weights stored in the memory 60 of the output signal calculating section 10 using an equation given by: EQU .DELTA.w.sub.ij =-.epsilon.*.differential.E/.differential.w.sub.ij +.alpha.*.DELTA.w'.sub.ij
where .SIGMA..sub.i is the sum associated with all the signal processing sections 30 at the uppermost layer in the output signal calculating section 10, .epsilon. is a positive constant called the "learning rate", .alpha. is a positive constant called an "acceleration parameter", and .DELTA.w'.sub.ij is the amount of alteration of the weights in the previous learning. In this way, the weights are altered.
The error can be made small by repeating the renewal of the weights. When the error becomes considerably small to the extent that the output signal is regarded as being satisfactorily close to a desired value, the learning is terminated. At this stage, the learning apparatus can recognize input characteristic data and can output a recognition result.
The learning apparatus having the above-described construction, however, must determine all the weights through a learning from a completely random state. Furthermore, upon completion of the learning, when the learning apparatus is required to learn a recognition operation so as to provide a desired output in response to unknown input data, a problem arises in that a time-consuming new learning is needed using previously learned data.